{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The `__new__` Method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've studied the `__init__` method quite a bit so far. It is basically a method that gets called right after the class instance has been created, usually invoked when we call the Class with arguments to instantiate an instance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `__new__` method is the method that is invoked to actually create the new object, as an instance of the desired class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the `object` class provides a default implementation for `__new__` we rarely have to bother with it, but sometimes we want to intercept the instance creation to tweak things a bit."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `__new__` method, unlike the `__init__` method is actually a **static** method, not an **instance** method. Which kinds of make sense since the instance does not exist yet - that's what the `__new__` method is trying to create.\n",
    "\n",
    "Why it's not a **class** method is more complicated. We'll see why that's the case as we explore `__new__`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember how we create instances of a class - we call the class with whatever arguments we need to initialize the class state:\n",
    "\n",
    "```\n",
    "p = Person(name, age)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The creation of the class instance is then done in two steps:\n",
    "1. The `__new__` method is called. It receives, as arguments, the class object we want an instance of, and any additional arguments we pass to the creation call (e.g. `name` and `age`). It should return a new instance of the class (and it may have used the arguments to initialize stuff in the class too, that's up to how you write your `__new__ method)\n",
    "2. If the object returned by `__new__` is an instance of the class specified in the call to `__new__`, then the `__init__` method is also called. The `__init__` method is an instance method and does not return anything (well, it returns None)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `__new__` method is present in the `object` class, so we can easily use it to create an instance of a class, without calling the class itself.\n",
    "\n",
    "Let's take a look:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Point:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = object.__new__(Point)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "__main__.Point"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So as you see, we created an instance of `Point` by using the `__new__` method defined in `object`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take that a step further and include the initialization as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Point:\n",
    "    def __init__(self, x, y):\n",
    "        self.x = x\n",
    "        self.y = y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = object.__new__(Point)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.__init__(10, 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'x': 10, 'y': 20}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p.__dict__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One thing to note is that although `object.__new__` will accept `*args` and `**kwargs` it does not actually use them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = object.__new__(Point, 10, 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p.__dict__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember that this automatic chaining of `__new__` and `__init__` happens when we create a class using it as a callable (e.g.  `Person(10, 20)`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, since `__new__` is just another method, we can choose to override it in our custom classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Point:\n",
    "    def __new__(cls, x, y):\n",
    "        print('Creating instance...', x, y)\n",
    "        instance = object.__new__(cls)  # delegate to object.__new__\n",
    "        return instance  # don't forget to return the new instance!\n",
    "    \n",
    "    def __init__(self, x, y):\n",
    "        print('Initializing instance...', x, y)\n",
    "        self.x = x\n",
    "        self.y = y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating instance... 10 20\n",
      "Initializing instance... 10 20\n"
     ]
    }
   ],
   "source": [
    "p = Point(10, 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's interesting also about the `__new__` method is that we can override it even when we inherit from the **built-in** types, whereas often the same does not work with `__init__` (we'll come back to the topic of inheriting from built-in types later when we look at abstract base classes.)\n",
    "\n",
    "Let's see an example of this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Squared(int):\n",
    "    def __new__(cls, x):\n",
    "        return super().__new__(cls, x**2)  # delegate creating an int instance to the int class itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = Squared(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Anf of course, the `type` of result is `Squared`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "__main__.Squared"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But, it is **also** an `int` since we **inherited** from `int`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "isinstance(result, int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trying to do this using `__init__` would not work - the built-in `__init__` for integers does not actually do anything, and does not allow for an argument to be passed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Squared(int):\n",
    "    def __init__(self, x):\n",
    "        print('calling init...')\n",
    "        super().__init__(x ** 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling init...\n",
      "object.__init__() takes exactly one argument (the instance to initialize)\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    result = Squared(4)\n",
    "except TypeError as ex:\n",
    "    print(ex)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most often when we override the `__new__` method we use delegation to the parent class to do some of the work. But of course, as we saw just now we don't have to, we can just use `object.__new__` directly. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's how we did it using `object` explicitly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __new__(cls, name):\n",
    "        print(f'Person: Instantiating {cls.__name__}...')\n",
    "        instance = object.__new__(cls)\n",
    "        return instance\n",
    "        \n",
    "    def __init__(self, name):\n",
    "        print(f'Person: Initializing instance...')\n",
    "        self.name = name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Person: Instantiating Person...\n",
      "Person: Initializing instance...\n"
     ]
    }
   ],
   "source": [
    "p = Person('Guido')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But the problem here is that this technique does not play well with inheritance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's do the same thing with a sub class of `Person`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Student(Person):\n",
    "    def __new__(cls, name, major):\n",
    "        print(f'Student: Instantiating {cls.__name__}...')\n",
    "        instance = object.__new__(cls)\n",
    "        return instance\n",
    "    \n",
    "    def __init__(self, name, major):\n",
    "        print(f'Student: Initializing instance...')\n",
    "        super().__init__(name)\n",
    "        self.major = major"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Student: Instantiating Student...\n",
      "Student: Initializing instance...\n",
      "Person: Initializing instance...\n"
     ]
    }
   ],
   "source": [
    "s = Student('John', 'Major')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll notice that the `__new__` method of `Person` was not called - that's because we called `object.__new__` directly.\n",
    "\n",
    "So instead we really should do it this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __new__(cls, name):\n",
    "        print(f'Person: Instantiating {cls.__name__}...')\n",
    "        instance = super().__new__(cls)\n",
    "        return instance\n",
    "        \n",
    "    def __init__(self, name):\n",
    "        print(f'Person: Initializing instance...')\n",
    "        self.name = name\n",
    "        \n",
    "class Student(Person):\n",
    "    def __new__(cls, name, major):\n",
    "        print(f'Student: Instantiating {cls.__name__}...')\n",
    "        instance = super().__new__(cls, name)\n",
    "        return instance\n",
    "    \n",
    "    def __init__(self, name, major):\n",
    "        print(f'Student: Initializing instance...')\n",
    "        super().__init__(name)\n",
    "        self.major = major"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Student: Instantiating Student...\n",
      "Person: Instantiating Student...\n",
      "Student: Initializing instance...\n",
      "Person: Initializing instance...\n"
     ]
    }
   ],
   "source": [
    "s = Student('John', 'Major')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So why override `__new__`? We saw one example where we can inherit from a built-in type and modify the behavior (the `Squared` class - the value is still an `int`, since we inherited from `int`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It allows us to tweak how the class is created. For example we could inject some extra attributes onto the class before creating the instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Square:\n",
    "    def __new__(cls, w, l):\n",
    "        cls.area = lambda self: self.w * self.l\n",
    "        # or use setattr(cls, 'area', lambda self: self.w * self.l)\n",
    "        instance = super().__new__(cls)  \n",
    "        return instance\n",
    "    \n",
    "    def __init__(self, w, l):\n",
    "        self.w = w\n",
    "        self.l = l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Square(3, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.area()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you see we injected a function into the class before creating it. We could also tweak the instance before returning it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Square:\n",
    "    def __new__(cls, w, l):\n",
    "        setattr(cls, 'area', lambda self: self.w * self.l)\n",
    "        instance = super().__new__(cls)\n",
    "        instance.w = w\n",
    "        instance.l = l\n",
    "        return instance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that since we are setting the instance variables inside the `__new__`, we don't even need to provide an override for the `__init__`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Square(3, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'w': 3, 'l': 4}"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.__dict__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.area()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keep in mind that `__new__` is a static method, and we can also invoke it explicitly ourselves - we just need to remember that we need to pass the class (type) we want to create an instance of to the `__new__` method as the first argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Square.__new__(Square, 3, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({'w': 3, 'l': 4}, 12)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.__dict__, s.area()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another important point. Remember that I said that when we call `MyClass(args, kwargs)`, it will essentially call:\n",
    "```\n",
    "MyClass.__new__(MyClass, args, kwargs)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But that's not the only thing that happens - the `__init__` is also automatically called right after.\n",
    "\n",
    "**But only if the type returned by `__new__` matches the type specified as the first argument of `__new__`**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __new__(cls, name):\n",
    "        print(f'Creating instance of {cls.__name__}... not really...')\n",
    "        instance = str(name)\n",
    "        return instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating instance of Person... not really...\n"
     ]
    }
   ],
   "source": [
    "p = Person('Alex')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('Alex', str)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p, type(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we requested a new instance of `Person`, but `__new__` ignored that and created an instance of `str` instead."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's add an init method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __new__(cls, name):\n",
    "        print(f'Creating instance of {cls.__name__}... not really...')\n",
    "        instance = str(name)\n",
    "        return instance\n",
    "    \n",
    "    def __init__(self, name):\n",
    "        print('Init called...')\n",
    "        self.name = name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating instance of Person... not really...\n"
     ]
    }
   ],
   "source": [
    "p = Person('Raymond')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(str, 'Raymond')"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(p), p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see the `__init__` was not called - and that makes sense - if `__new__` is not returning an instance of `Person` it does not make sense to invoke the `__init__` for `Person`, nor for the newly created instance (the signature might not even be compatible!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One more point to make, is that if we override the `__new__` method, there is probably no reason to also override the `__init__` method, since we can take care of any custom initialization in the `__new__` method ourselves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __new__(cls, name, age):\n",
    "        instance = super().__new__(cls)\n",
    "        instance.name = name\n",
    "        instance.age = age\n",
    "        return instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = Person('Guido', 42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'Guido', 'age': 42}"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p.__dict__"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
